昨天透過External Network, OVN 使用SNAT讓VM可以連線到Internet上，但是目前還是無法由Internet連線到VM。如果要由Internet直接連線到VM，則必需建立一個floating IP，讓Internet上的流量由OVN透過DNAT連線到VM。這部份的概念在Day-07: 透過SNAT與DNAT 連至Internet的DNAT內容也有說明過。在開始今天的內容之前，可以先回去複習。
延續昨天的實驗架構，今天與昨天的架構是完全一致的。只是今天會接著在External Network上申請floating IP給VM使用。
openstack network create --provider-network-type flat --provider-physical-network extnet0 ext_n2 \
--external
SUBNET_RANGE=10.0.2.0/24
GATEWAY=10.0.2.2
ALLOCATION_RANGE=start=10.0.2.210,end=10.0.2.220
openstack subnet create --subnet-range ${SUBNET_RANGE} --network ext_n2 extsubnet \
--no-dhcp \
--gateway ${GATEWAY} \
--allocation-pool ${ALLOCATION_RANGE}
可以注意到取得的Floating IP 會屬於建立External Netwrok時，指定的 --allocation-pool之間。
openstack floating ip create ext_n2
FLOATING_IP=$(openstack floating ip list --format json | jq -rc '.[]."Floating IP Address"')
openstack server add floating ip vm_1 ${FLOATING_IP}
# Port
openstack port list --long -c ID -c "Fixed IP Addresses" -c "Device Owner" | abbrev
+--------+-------------------------------------------------+--------------------------+
| ID | Fixed IP Addresses | Device Owner |
+--------+-------------------------------------------------+--------------------------+
| 0bece6 | ip_address='172.16.100.10' , subnet_id='ec52a9' | compute:nova |
| 06ff4f | ip_address='172.16.100.2' , subnet_id='ec52a9' | network:distributed |
| 1c52f2 | ip_address='172.16.100.1' , subnet_id='ec52a9' | network:router_interface |
| e70e83 | | network:distributed |
| 268c32 | ip_address='10.0.2.211' , subnet_id='6f6c62' | network:router_gateway |
| cbae0e | ip_address='10.0.2.215' , subnet_id='6f6c62' | network:floatingip |
+--------+-------------------------------------------------+--------------------------+
要使用Floating IP存取VM時，必需建立security group設定允許的rule。實驗環境已經事先建立一個允許ICMP與SSH的sg，直接將其套用至vm上。如果沒有相對應的sg，外部會無法進行連線。
# show the pre-created sg
openstack security group show allow-icmp-ssh
# add sg to instances
openstack server add security group vm_1 allow-icmp-ssh
建立floating IP後，在logical router上，除了昨天看到的SNAT規則外，也可以查到DNAT的設定，表示logical router會將來自 10.0.2.215的外部IP與內部的172.16.100.10，做DNAT而達到與網際網路互連。
router_name=$(ovn-nbctl --format json --columns name list logical_router | jq -cr .data[][])
ovn-nbctl lr-nat-list $router_name
TYPE GATEWAY_PORT EXTERNAL_IP EXTERNAL_PORT LOGICAL_IP EXTERNAL_MAC LOGICAL_PORT
dnat_and_snat 10.0.2.215 172.16.100.10
snat 10.0.2.211 172.16.100.0/24
ovn-nbctl show $router_name| abbrev
router 1a0898 (neutron-ac7c63) (aka r)
port lrp-91be9b
mac: "fa:16:3e:d0:74:be"
networks: ["172.16.100.1/24"]
port lrp-6fd696
mac: "fa:16:3e:84:56:22"
networks: ["192.168.121.211/24"]
gateway chassis: [3caa2f]
nat 643964
external ip: "10.0.2.211"
logical ip: "172.16.100.0/24"
type: "snat"
nat c92c5b
external ip: "10.0.2.215"
logical ip: "172.16.100.10"
type: "dnat_and_snat"
我們的實驗環境是透用Vagrant和VirtualBox建立OpenStack環境。建立的VM對外連線都是依靠eth0，而VM的eth0是採用VirtualBox的NAT網路模式。在這個模式下，每個VM都有自己的NAT網路與gateway，所以每個VM是無法透過eth0互相通信。對今天的實驗來說，在Network-Controller 的eth0上的網路，就等同於Internet，所以我們會在Network-Controller上進行Floating IP的連通性驗證。
在 VM內執行 ping 8.8.8.8
可以看到ICMP request 是經由geneve tunnel 送至network-controller；稍後的ICMP reply也是經由geneve tunnel 送回來。
tcpdump -vvneei eth1 'udp port 6081'
03:47:28.806777 52:54:00:ea:65:ed > 52:54:00:82:23:6d, ethertype IPv4 (0x0800), length 156: (tos 0x0, ttl 64, id 19294, offset 0, flags [DF], proto UDP (17), length 142)
192.168.33.20.51362 > 192.168.33.10.geneve: [bad udp cksum 0xc3fa -> 0x948a!] Geneve, Flags [C], vni 0x3, proto TEB (0x6558), options [class Open Virtual Networking (OVN) (0x102) type 0x80(C) len 8 data 00030002]
fa:16:3e:84:56:22 > 52:54:00:f9:f0:f0, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 54783, offset 0, flags [DF], proto ICMP (1), length 84)
172.16.100.10 > 8.8.8.8: ICMP echo request, id 18017, seq 1, length 64
03:47:28.813100 52:54:00:82:23:6d > 52:54:00:ea:65:ed, ethertype IPv4 (0x0800), length 156: (tos 0x0, ttl 64, id 51784, offset 0, flags [DF], proto UDP (17), length 142)
192.168.33.10.nfoldman > 192.168.33.20.geneve: [bad udp cksum 0xc3fa -> 0x0ae1!] Geneve, Flags [C], vni 0x1, proto TEB (0x6558), options [class Open Virtual Networking (OVN) (0x102) type 0x80(C) len 8 data 00030002]
fa:16:3e:d0:74:be > fa:16:3e:3c:24:6e, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 108, id 0, offset 0, flags [none], proto ICMP (1), length 84)
8.8.8.8 > 172.16.100.10: ICMP echo reply, id 18017, seq 1, length 64
network-controller 由geneve tunnel 收到ICMP request後，會由eth0 對外送至Internet. 特別注意，從eth0 出去時，source IP已經透過 DNAT換成10.0.2.215. 之後由Internet收到給10.0.2.215的ICMP reply，經由DNAT轉換後，在geneve tunnel上會看到ICMP reply是回給 172.16.100.10。
tcpdump -vvneei eth1 'udp port 6081'
03:47:28.806598 52:54:00:ea:65:ed > 52:54:00:82:23:6d, ethertype IPv4 (0x0800), length 156: (tos 0x0, ttl 64, id 19294, offset 0, flags [DF], proto UDP (17), length 142)
192.168.33.20.51362 > 192.168.33.10.geneve: [bad udp cksum 0xc3fa -> 0x948a!] Geneve, Flags [C], vni 0x3, proto TEB (0x6558), options [class Open Virtual Networking (OVN) (0x102) type 0x80(C) len 8 data 00030002]
fa:16:3e:84:56:22 > 52:54:00:f9:f0:f0, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 54783, offset 0, flags [DF], proto ICMP (1), length 84)
172.16.100.10 > 8.8.8.8: ICMP echo request, id 18017, seq 1, length 64
03:47:28.812439 52:54:00:82:23:6d > 52:54:00:ea:65:ed, ethertype IPv4 (0x0800), length 156: (tos 0x0, ttl 64, id 51784, offset 0, flags [DF], proto UDP (17), length 142)
192.168.33.10.nfoldman > 192.168.33.20.geneve: [bad udp cksum 0xc3fa -> 0x0ae1!] Geneve, Flags [C], vni 0x1, proto TEB (0x6558), options [class Open Virtual Networking (OVN) (0x102) type 0x80(C) len 8 data 00030002]
fa:16:3e:d0:74:be > fa:16:3e:3c:24:6e, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 108, id 0, offset 0, flags [none], proto ICMP (1), length 84)
8.8.8.8 > 172.16.100.10: ICMP echo reply, id 18017, seq 1, length 64
tcpdump -i eth0 icmp
03:47:28.807129 IP 10.0.2.215 > dns.google: ICMP echo request, id 18017, seq 1, length 64
03:47:28.812197 IP dns.google > 10.0.2.215: ICMP echo reply, id 18017, seq 1, length 64
在network-controller上執行 ping 10.0.2.215，雖然是要連到10.0.2.215，但在geneve tunnel上，會發現已經透過DNAT換成要送給172.16.100.10。
tcpdump -vvneei eth1 'udp port 6081'
03:52:14.742209 52:54:00:82:23:6d > 52:54:00:ea:65:ed, ethertype IPv4 (0x0800), length 156: (tos 0x0, ttl 64, id 39673, offset 0, flags [DF], proto UDP (17), length 142)
192.168.33.10.47005 > 192.168.33.20.geneve: [bad udp cksum 0xc3fa -> 0x7024!] Geneve, Flags [C], vni 0x1, proto TEB (0x6558), options [class Open Virtual Networking (OVN) (0x102) type 0x80(C) len 8 data 00030002]
fa:16:3e:d0:74:be > fa:16:3e:3c:24:6e, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 12034, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.2.15 > 172.16.100.10: ICMP echo request, id 1, seq 1, length 64
03:52:14.744315 52:54:00:ea:65:ed > 52:54:00:82:23:6d, ethertype IPv4 (0x0800), length 156: (tos 0x0, ttl 64, id 52973, offset 0, flags [DF], proto UDP (17), length 142)
192.168.33.20.5380 > 192.168.33.10.geneve: [bad udp cksum 0xc3fa -> 0x1825!] Geneve, Flags [C], vni 0x3, proto TEB (0x6558), options [class Open Virtual Networking (OVN) (0x102) type 0x80(C) len 8 data 00030002]
fa:16:3e:84:56:22 > de:0d:fe:eb:97:48, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 48875, offset 0, flags [none], proto ICMP (1), length 84)
172.16.100.10 > 10.0.2.15: ICMP echo reply, id 1, seq 1, length 64
在compute-01上，會收到透過geneve tunnel由network-controller送過來的 ICMP request；之後，由geneve tunnel 送ICMP reply 回給network-controller。
tcpdump -vvneei eth1 'udp port 6081'
03:52:14.742622 52:54:00:82:23:6d > 52:54:00:ea:65:ed, ethertype IPv4 (0x0800), length 156: (tos 0x0, ttl 64, id 39673, offset 0, flags [DF], proto UDP (17), length 142)
192.168.33.10.47005 > 192.168.33.20.geneve: [bad udp cksum 0xc3fa -> 0x7024!] Geneve, Flags [C], vni 0x1, proto TEB (0x6558), options [class Open Virtual Networking (OVN) (0x102) type 0x80(C) len 8 data 00030002]
fa:16:3e:d0:74:be > fa:16:3e:3c:24:6e, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 12034, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.2.15 > 172.16.100.10: ICMP echo request, id 1, seq 1, length 64
03:52:14.744384 52:54:00:ea:65:ed > 52:54:00:82:23:6d, ethertype IPv4 (0x0800), length 156: (tos 0x0, ttl 64, id 52973, offset 0, flags [DF], proto UDP (17), length 142)
192.168.33.20.5380 > 192.168.33.10.geneve: [bad udp cksum 0xc3fa -> 0x1825!] Geneve, Flags [C], vni 0x3, proto TEB (0x6558), options [class Open Virtual Networking (OVN) (0x102) type 0x80(C) len 8 data 00030002]
fa:16:3e:84:56:22 > de:0d:fe:eb:97:48, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 48875, offset 0, flags [none], proto ICMP (1), length 84)
172.16.100.10 > 10.0.2.15: ICMP echo reply, id 1, seq 1, length 64
其實，OpenStack 的Floating IP的原理，是直接使用OVN的DNAT機制，所以和Day-07: 透過SNAT與DNAT 連至Internet相比，是不是非常相似呢?
iThome鐵人賽
看影片追技術